FastDeepIoT: Towards Understanding and Optimizing Neural Network Execution Time on Mobile and Embedded Devices

Deep neural networks show great potential as solutions to many sensing application problems, but their excessive resource demand slows down execution time, pausing a serious impediment to deployment on low-end devices. To address this challenge, recent literature focused on compressing neural network size to improve performance. We show that changing neural network size does not proportionally affect performance attributes of interest, such as execution time. Rather, extreme run-time nonlinearities exist over the network configuration space. Hence, we propose a novel framework, called FastDeepIoT, that uncovers the non-linear relation between neural network structure and execution time, then exploits that understanding to find network configurations that significantly improve the trade-off between execution time and accuracy on mobile and embedded devices. FastDeepIoT makes two key contributions. First, FastDeepIoT automatically learns an accurate and highly interpretable execution time model for deep neural networks on the target device. This is done without prior knowledge of either the hardware specifications or the detailed implementation of the used deep learning library. Second, FastDeepIoT informs a compression algorithm how to minimize execution time on the profiled device without impacting accuracy. We evaluate FastDeepIoT using three different sensing-related tasks on two mobile devices: Nexus 5 and Galaxy Nexus. FastDeepIoT further reduces the neural network execution time by $48\%$ to $78\%$ and energy consumption by $37\%$ to $69\%$ compared with the state-of-the-art compression algorithms.Comment: Accepted by SenSys '1

Voice Pitch Elicited Frequency Following Response in Chinese Elderlies

Background: Perceptual and electrophysiological studies have found reduced speech discrimination in quiet and noisy environment, delayed neural timing, decreased neural synchrony, and decreased temporal processing ability in elderlies, even those with normal hearing. However, recent studies have also demonstrated that language experience and auditory training enhance the temporal dynamics of sound encoding in the auditory brainstem response. The purpose of this study was to explore the pitch processing ability at the brainstem level in an aging population that has a tonal language background.Method: Mandarin speaking younger (n=12) and older (n=12) adults were recruited for this study. All participants had normal audiometric test results and normal suprathreshold click-evoked auditory brainstem responses (ABR). To record Frequency Following Responses (FFR) elicited by Mandarin lexical tones, two Mandarin Chinese syllables with different fundamental frequency pitch contours (Flat Tone and Falling Tone) were presented at 70 dB SPL. Fundamental frequencies (f0) of both the stimulus and the responses were extracted and compared to individual brainstem responses. Two indices were used to examine different aspects of pitch processing ability at the brainstem level: Pitch Strength and Pitch Correlation. Results: Lexical tone elicited FFR were overall weaker in the older adult group compared to their younger adult counterpart. Measured by Pitch Strength and Pitch Correlation, statistically significant group differences were only found when the tone with a falling f0 (Falling Tone) were used as the stimulus.Conclusion: Results of this study demonstrated that in a tonal language speaking population, pitch processing ability at the brainstem level of older adults are not as strong and robust as their younger counterparts. Findings of this study are consistent with previous reports on brainstem responses of older adults whose native language is English. On the other hand, lexical tone elicited FFRs have been shown to correlate with the length of language exposure. Older adults’ degraded responses in our study may also be due to that, the Mandarin speaking older adults’ long term exposure somewhat counteracted the negative impact on aging and helped maintain, or at least reduced, the degradation rate in their temporal processing capacity at the brainstem level

Voice pitch elicited frequency following response in Chinese elderlies

Background: Perceptual and electrophysiological studies have found reduced speech discrimination in quiet and noisy environment, delayed neural timing, decreased neural synchrony, and decreased temporal processing ability in elderlies, even those with normal hearing. However, recent studies have also demonstrated that language experience and auditory training enhance the temporal dynamics of sound encoding in the auditory brainstem response (ABR). The purpose of this study was to explore the pitch processing ability at the brainstem level in an aging population that has a tonal language background. Method: Mandarin speaking younger (n = 12) and older (n = 12) adults were recruited for this study. All participants had normal audiometric test results and normal suprathreshold click-evoked ABR. To record frequency following responses (FFRs) elicited by Mandarin lexical tones, two Mandarin Chinese syllables with different fundamental frequency pitch contours (Flat Tone and Falling Tone) were presented at 70 dB SPL. Fundamental frequencies (f0) of both the stimulus and the responses were extracted and compared to individual brainstem responses. Two indices were used to examine different aspects of pitch processing ability at the brainstem level: Pitch Strength and Pitch Correlation. Results: Lexical tone elicited FFR were overall weaker in the older adult group compared to their younger adult counterpart. Measured by Pitch Strength and Pitch Correlation, statistically significant group differences were only found when the tone with a falling f0 (Falling Tone) were used as the stimulus. Conclusion: Results of this study demonstrated that in a tonal language speaking population, pitch processing ability at the brainstem level of older adults are not as strong and robust as their younger counterparts. Findings of this study are consistent with previous reports on brainstem responses of older adults whose native language is English. On the other hand, lexical tone elicited FFRs have been shown to correlate with the length of language exposure. Older adults\u27 degraded responses in our study may also be due to that, the Mandarin speaking older adults\u27 long term exposure somewhat counteracted the negative impact on aging and helped maintain, or at least reduced, the degradation rate in their temporal processing capacity at the brainstem level

Effects of Inter-Stimulus Interval on Speech-Evoked Frequency-Following Response in Elderly Adults

Background: The speech-evoked frequency following response (FFR) has shown to be useful in assessing complex auditory processing abilities and in different age groups. While many aspects of FFR have been studied extensively, the effect of timing, as measured by inter-stimulus-interval (ISI), especially in the older adult population, has yet to be thoroughly investigated. Objective: The purpose of this study was to examine the effects of different ISIs on speech evoked FFR in older and younger adults who speak a tonal language, and to investigate whether the older adults’ FFR were more susceptible to the change in ISI. Materials and Methods: Twenty-two normal hearing participants were recruited in our study, including 11 young adult participants and 11 elderly participants. An Intelligent Hearing Systems Smart EP evoke potential system was used to record the FFR in four ISI conditions (40, 80, 120 and 160 ms). A recorded natural speech token with a falling tone /yi/ was used as the stimulus. Two indices, stimulus-to-response correlation coefficient and pitch strength, were used to quantify the FFR responses. Two-way analysis of variance (ANOVA) was used to analyze the differences in different age groups and different ISI conditions. Results: There was no significant difference in stimulus-to-response correlation coefficient and pitch strength among the different ISI conditions, in either age groups. Older adults appeared to have weaker FFR for all ISI conditions when compared to their younger adult counterparts. Conclusion: Shorter ISIs did not result in worse FFRs from older adults or younger adults. For speech-evoked FFR using a recorded natural speech token that is 250 ms in length, an ISI of as short as 40 ms appeared to be sufficient and effective to record FFR for elderly adults

FOCAL: Contrastive Learning for Multimodal Time-Series Sensing Signals in Factorized Orthogonal Latent Space

This paper proposes a novel contrastive learning framework, called FOCAL, for extracting comprehensive features from multimodal time-series sensing signals through self-supervised training. Existing multimodal contrastive frameworks mostly rely on the shared information between sensory modalities, but do not explicitly consider the exclusive modality information that could be critical to understanding the underlying sensing physics. Besides, contrastive frameworks for time series have not handled the temporal information locality appropriately. FOCAL solves these challenges by making the following contributions: First, given multimodal time series, it encodes each modality into a factorized latent space consisting of shared features and private features that are orthogonal to each other. The shared space emphasizes feature patterns consistent across sensory modalities through a modal-matching objective. In contrast, the private space extracts modality-exclusive information through a transformation-invariant objective. Second, we propose a temporal structural constraint for modality features, such that the average distance between temporally neighboring samples is no larger than that of temporally distant samples. Extensive evaluations are performed on four multimodal sensing datasets with two backbone encoders and two classifiers to demonstrate the superiority of FOCAL. It consistently outperforms the state-of-the-art baselines in downstream tasks with a clear margin, under different ratios of available labels. The code and self-collected dataset are available at https://github.com/tomoyoshki/focal.Comment: Code available at: [github](https://github.com/tomoyoshki/focal

Learning a second language in adulthood changes subcortical neural encoding

Second language learning has been shown to impact and reshape the central nervous system, anatomically and functionally. Most of the studies on second language learning and neuroplasticity have been focused on cortical areas, whereas the subcortical neural encoding mechanism and its relationship with L2 learning have not been examined extensively. The purpose of this study was to utilize frequency-following response (FFR) to examine if and how learning a tonal language in adulthood changes the subcortical neural encoding in hearing adults. Three groups of subjects were recruited: native speakers of Mandarin Chinese (native speakers (NS)), learners of the language (L2 learners), and those with no experience (native speakers of foreign languages (NSFL)). It is hypothesized that differences would exist in FFRs obtained from the three language experience groups. Results revealed that FFRs obtained from L2 learners were found to be more robust than the NSFL group, yet not on a par with the NS group. Such results may suggest that in human adulthood, subcortical neural encoding ability may be trainable with the acquisition of a new language and that neuroplasticity at the brainstem level can indeed be influenced by L2 learning

Eugene: Towards deep intelligence as a service

National Research Foundation (NRF) Singapore under its International Research Centres in Singapore Funding Initiativ

Derivation and Characterization of Hepatic Progenitor Cells from Human Embryonic Stem Cells

The derivation of hepatic progenitor cells from human embryonic stem (hES) cells is of value both in the study of early human liver organogenesis and in the creation of an unlimited source of donor cells for hepatocyte transplantation therapy. Here, we report for the first time the generation of hepatic progenitor cells derived from hES cells. Hepatic endoderm cells were generated by activating FGF and BMP pathways and were then purified by fluorescence activated cell sorting using a newly identified surface marker, N-cadherin. After co-culture with STO feeder cells, these purified hepatic endoderm cells yielded hepatic progenitor colonies, which possessed the proliferation potential to be cultured for an extended period of more than 100 days. With extensive expansion, they co-expressed the hepatic marker AFP and the biliary lineage marker KRT7 and maintained bipotential differentiation capacity. They were able to differentiate into hepatocyte-like cells, which expressed ALB and AAT, and into cholangiocyte-like cells, which formed duct-like cyst structures, expressed KRT19 and KRT7, and acquired epithelial polarity. In conclusion, this is the first report of the generation of proliferative and bipotential hepatic progenitor cells from hES cells. These hES cell–derived hepatic progenitor cells could be effectively used as an in vitro model for studying the mechanisms of hepatic stem/progenitor cell origin, self-renewal and differentiation